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PERSONALIZED MOTION IMAGING SYSTEM 



FIELD OF THE INVENTION 

The present invention relates to imaging systems that incorporate 
5 personal images in computer generated image sequences. More particularly the 
invention relates to such systems that include a camera for capturing an image of a 
viewer and incorporate the captured image into a computer generated motion 
image sequence. 

1 0 BACKGROUND OF THE INVENTION 

Digital motion imaging systems are traditionally used for both 
passive and interactive viewing. In the former case, pre-defined image sequences 
are presented to a viewer by, for example, a digital television or digital cinema 
experience. In the latter C£ise, choices made by the viewer affect the images 

1 5 presented, for example in computer gaming. Passive viewing experiences are 
often created by mixing pre-stored image sequences (on videotape or film, for 
example) to create the experiences. Live (real-time) motion image sequences are 
often passively viewed as part of the electronic media, for example with sporting 
events, news shows, and the like. In contrast, interactive viewing experiences are 

20 commonly found as part of computer gaming activities in which a computer 

generates a scene that is generally rendered from a database of objects describing 
the scene. Viewer interaction through some interface device (such as a joystick or 
mouse) provides additional information for suitably rendering the scene. 

The integration of images from multiple sources is often found as 

25 part of image sequence viewing experiences, both passive and interactive. For 
example, multiple scenes are presented within a larger scene or multiple image 
sequence sources are viewed simultaneously on a single image-sequence viewing 
device. However, these image sequences are not personalized, that is they do not 
include real-time motion image sequences of the viewers. Commercial experience 

30 has shown, for example with the Kodak Fantasy Theater and Thrill Ride imaging 
systems, that personalized images have a greater appeal to many image viewers 
than images without personalization. Moreover, for some applications, the 



integration of an image of the viewer himself or herself with a computer generated 
image sequence adds to the enjoyment of the viewing experience, for example 
with group gaming activities. In these applications, the presentation of 
participants within the image sequences viewed adds verisimilitude and additional 
enjoyment to the experience. See for example, US Patent 4,710,873, issued 
December 1, 1987 to Breslow et al. which discloses incorporating a video still 
image of a viewer into an image sequence in an interactive display device. It is 
also known to provide interaction between multiple viewers within some imaging 
environments to enhance the enjoyment of viewers and improve the effectiveness 
of the imaging experience. See for example US Patent 4,521,014 issued June 4, 
1 985 to Sitrick. The problem with these approaches is that the image of the 
viewer is static, thereby reducing the impact of the effect. 

There is a need therefore for an improved real-time motion imaging 
system that provides personalization, improves communication, and does not 
require preparation. 

SUMMARY OF THE INVENTION 

The need is met according to the present invention by providing a 
personalized motion imaging system that includes: a computer; a motion image 
display device connected to the computer for displaying a motion image sequence 
to a viewer; a camera connected to the computer for capturing an image of the 
viewer; and an image processing program running on the computer for generating 
a real-time motion image sequence of an environment and integrating a motion 
image sequence of the viewer into the computer generated motion image sequence 
of the environment to produce a composite motion image sequence for display on 
a motion image display device. 

ADVANTAGES 

The present invention has the advantage that it provides a real-time 
motion imaging system that provides personalization, improves communication, 
and does not require preparation. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic diagram showing an improved real-time 
motion imaging system according to the present invention; and 

Fig. 2 is a schematic diagram of a system having a plurality of 
5 personalized motion imaging systems according to the present invention. 

DETAILED DESCRIPTION OF THE INVENTION 

Referring to Fig. 1, a personalized motion imaging system 10 
includes a computer 12, a motion image sequence display device 14 connected to 

10 the computer, and a camera 16, such as a motion image camera, also connected to 
the computer. A variety of programs for compositing and animating image 
sequences can run on the computer. 

The composition and animation of image sequences can operate in 
a variety of ways. In a first embodiment, a viewer 17 stations herself in front of 

1 5 the viewing device 14 and observes a motion image sequence generated by the 
computer 12. The camera 16 records the viewer in real-time and transmits an 
image sequence of the viewer to the computer 12 . The computer 12 then 
composites the image sequence of the viewer with a computer generated image 
sequence to create a personalized, real-time image sequence and displays the 

20 personalized composite image sequence to the viewer. Known composition 
techniques using a blue or green screen to differentiate the subject from the 
background may be employed. See for example US Patent 5,867,282 issued 
February 2, 1999 by Fredlund et al. 

In an alternative embodiment, a still image of the viewer is 

25 captured by the camera 16 and transmitted to the computer 12. The computer then 
creates an animated image of the viewer using the still image and an animation 
routine. This animated image is then composited with the computer generated 
image sequence to form the personalized composite image sequence. 

In either embodiment described above, the computer generated 

30 image sequence is generated from a stored set of image objects. The computer 
generated image sequence content is structured so as to complement the captured 
or animated personal image sequence. The computer generated image sequence is 
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provided with locations in the individual images of the computer generated 
sequence where the individual images from the personalized image sequence or 
animation can be located so as to provide a suitable artistic and enjoyable 
sequence for the viewer. For example, in a game with opposing players, each of 
5 whom is controlling a virtual vehicle, the position of the vehicle operator can be 
fixed within the vehicle and a personalized image sequence composited into the 
vehicle operator position. 

The present invention has applications for both single viewer 
environments and environments in which there are multiple viewers. For 

10 example, in the single viewer case a viewer sees an image of himself integrated 
into the image sequence he views. Referring to Fig. 2, in the multiple viewer 
case, a viewer 17 is seen by a second viewer 17' viewing a related scene displayed 
on a second personalized motion imaging system 10'. Each viewer sees the other 
person and/or herself in the image sequence. Both of these cases are applicable to 

1 5 computer game playing with either single or multiple players. The multiple 

viewer case can be extended to more than two viewers to the extent supported by 
the computer and the image sequences. 

The verisimilitude of the imaging environment can be greatly 
enhanced with the use of stereo imagery and a stereoscopic viewing device. A 

20 computer generated stereo image sequence is composited with a personalized 
stereo image sequence to produce a stereo composite image sequence. The 
personalized stereo image can be acquired with the use of a stereo camera; 
multiple image acquisition of the viewer from different angles using a single 
camera; or by the generation of a stereo animation from either single or multiple 

25 still image(s) as known in the art. 

The composite stereo image sequence may be viewed in a variety 
of ways. Stereo image goggles can be used, for example, but the viewer will 
appear in the image with the goggles. For conventional viewing without goggles, 
a monocentric autostereoscopic display device such as that disclosed in copending 

30 US Serial No. 09/738,747 filed December 15, 2000 by Kessler et al. may be used. 
Alternatively, other stereo display devices known in the art may be employed. 
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The personalized display system may be extended to include other 
means of interaction in a single- or multi-viewer environment. For example, a 
multi-sensory experience can be provided by including audio speakers 15 to 
provide sound to the viewer. These sounds are typically associated with the image 
5 sequence being viewed, as in a motion picture or a computer game with sound 
effects. In a multi-viewer environment, the sound effects can be augmented by 
verbal interactions between players. These verbal interactions can be 
implemented with the use of microphones 18 and audio mixing devices (either 
separate or integrated into the computer) for incorporating the verbal interactions 

1 0 into the image sequence sound track or sound effects. 

In some applications, live verbal interactions between multiple 
viewers may not be desired. For example, a personalized representation of a 
viewer may utter pre-stored speech sequences stored within the computer. These 
speech sequences may be stored as digitized audio clips or synthesized in real- 

1 5 time from stored text. In any case, to the extent that the personalized image 

sequence, whether animated or not, can be seen to speak, the speech sequences are 
synchronized with the personalized image sequence. This capability is known in 
the prior art, for example with the RealSpeak product from Lemout & Houspie, 
and the animation products from LifeFX Networks, Inc. 

20 It is also possible to add additional verisimilitude to an animated, 

personalized image sequence by artificially animating emotions corresponding to 
the content of the image sequence. A variety of facial expressions are known to 
connote particular expressions; by animating personalized facial expressions at 
appropriate moments, viewers may further enjoy the viewing experience and 

25 interaction with other viewers. Such emotional animation is known in the art, for 
example by LifeFX Networks, Inc. 

A viewer's enjoyment of the imaging experience available through 
the present invention can be further enhanced with the addition of mementos of 
the experience. These can be provided by connecting an image output device 20 

30 to the computer 12. This image output device 20 might write any of a variety of 
images on a variety of media, such as traditional hard-copy images, three- 
dimensional lenticular images, image sequences on videotape or digital versatile 
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disks (DVD) 22, and the like. The image content may be drawn from the 
composited image sequence and may include multiple viewers and text 
reminiscent of the viewing experience. For example, a hardcopy print might be 
provided that includes a representative image from the composited sequence, 
5 suitable branding identification, personal images, and text describing the name of 
the experience and any scoring achievements. As another example, a DVD might 
be provided with the digital image sequence written thereon together with images 
of all of the participants. As another example, a lenticular image print might be 
provided that includes several selected scenes from the sequence incorporating 
10 personal images and experience identification. See for example US Patent 
5,764,23 1 issued June 9, 1998 to Ray at al. 

The invention has been described in detail with particular reference 
to certain preferred embodiments thereof, but it will be understood that variations 
and modifications can be effected within the spirit and scope of the invention. 
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PARTS LIST 
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